Binocular image alignment for near-eye display

ABSTRACT

A near-eye display device comprises a left-eye optical system and a right-eye optical system. Each of the left-eye optical system and the right-eye optical system comprises a holographic optical component positioned in a field of view of a user eye, an image source configured to emit imaging light, and an alignment optical component, wherein projection beam path between the image source and the light-deflecting optical component and an alignment beam path between the alignment optical component and the light-deflecting component share a common optical path.

BACKGROUND

A near-eye display device may provide virtual images to a user's eye toprovide an immersive augmented-reality experience. A virtual image maybe provided in various ways. In one example, a near-eye display devicemay include an image source configured to project an image along anoptical path via one or more optical components to a user's eye.

SUMMARY

Examples related to the binocular alignment of near-eye display devicesare disclosed. In one example, a near-eye display device comprises aleft-eye optical system and a right-eye optical system. Each of theleft-eye optical system and the right-eye optical system comprises alight-deflecting optical component positioned in a field of view of auser eye, an image source configured to emit imaging light, and analignment optical component, wherein a projection beam path between theimage source and the light-deflecting optical component and an alignmentbeam path between the alignment optical component and thelight-deflecting component share a common optical path.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example near-eye display device.

FIGS. 2A-2B shows an example optical system for a near-eye displaydevice.

FIG. 3 shows a near-eye display device including left-eye and right-eyealignment optical components in the form of visible-light camerasdirected to acquire images of a physical space.

FIG. 4 shows a near-eye display device including left-eye and right-eyealignment optical components in the form of visible-light camerasdirected to acquire images of a user.

FIG. 5 shows a near-eye display device including left-eye and right-eyealignment optical components in the form of infrared cameras directed toacquire images of a physical space.

FIG. 6 shows a near-eye display device including left-eye and right-eyealignment optical components in the form of infrared cameras directed toacquire images of a user.

FIG. 7 shows a near-eye display device including left-eye and right-eyealignment optical components in the form of infrared projectorsconfigured to emit different patterns of infrared light towards aphysical space.

FIG. 8 shows a near-eye display device including left-eye and right-eyealignment optical components in the form of infrared projectorsconfigured to emit different patterns of infrared light towards a user.

FIGS. 9A-9B show a near-eye display device including left-eye andright-eye alignment optical components in the form of infraredprojectors configured to emit infrared light according to a timemultiplexing scheme.

FIG. 10 shows an example method for aligning left and right eye images.

FIG. 11 shows an example computing system.

FIGS. 12A-12B show another example optical system for a near-eye displaydevice.

DETAILED DESCRIPTION

A binocular near-eye display device may project separate virtual imagesto each of a left eye and a right eye of a user to provide an immersiveaugmented-reality experience. As described in more detail below, abinocular near-eye display device may include separate optical systemsfor each of the left eye and the right eye of the user, wherein eachoptical system includes an image source configured to project a virtualimage through an optical component (e.g., a light guide). The opticalcomponent directs the virtual image to a volume hologram, which directsthe virtual image toward a user's eye.

In such a device, there is a risk that left-eye and right-eye virtualimages may not be properly aligned. For example, the separate left-eyeand right-eye optical systems may become misaligned due to one or moreoptical components in either of the optical systems bending, twisting,or otherwise deforming. In a device with a flexible frame, such opticalcomponents may deform during placement of the near-eye display device onthe user's head, and/or in other situations. Such misalignment of thevirtual images projected to each of the left eye and right eye, even bya fraction of a degree, may lead to the misaligned presentation ofstereoscopic images.

Accordingly, examples are disclosed that relate to calibrating separateleft-eye and right-eye optical systems of a binocular, near-eye displaydevice to align virtual images projected separately to a left eye and aright eye of a user. As described in more detail below, each of theleft-eye optical system and the right-eye optical system includesoptical components arranged to form a common optical path along whichboth image light and alignment light travel. The image light is used toform a virtual image, and the alignment light is used to calibrate theleft-eye and right-eye optical systems of the binocular near-eye displaydevice to align a left-eye image provided to a left eye of the user witha right-eye image provided to a right eye of the user. Because both theimage light and the alignment light use a common optical path in each ofthe optical systems, deformation of the common optical path affects theimage light and the alignment light identically for the optical system.Accordingly, images provided by the left-eye optical system and theright-eye optical system may be aligned relative to each other evenwhere either of the left-eye or right-eye optical path becomes deformed.

FIG. 1 shows aspects of a binocular near-eye display device 100. Thedisplay device 100 includes right-eye and left-eye optical systems 102Rand 102L mounted to a frame 104 configured to rest on a wearer's head.Each of the right-eye and left-eye optical systems 102 includelight-deflecting image display componentry configured to projectcomputerized virtual imagery into left and right display windows 106Rand 106L in the wearer's field of view (FOY). In one example, thelight-deflecting image display componentry includes one or moreholographic optical components configured to deflect image light. Anexample optical system 200 representative of the right-eye and left-eyeoptical systems 102R and 102L is described in more detail below withreference to FIGS. 2A-2B and 3-9.

In some implementations, the right and left display windows 106R and106L are wholly or partially transparent from the perspective of thewearer, to give the wearer a clear view of his or her surroundings. Insome implementations, the right and left display windows 106R, 106L areopaque, such that the wearer is completely absorbed in thevirtual-reality (VR) imagery provided via the near-eye display device.In some implementations, the opacity of the right and left displaywindows 106R, 106L is controllable dynamically via a dimming filter. Asubstantially see-through display window, accordingly, may be switchedto full opacity for a fully immersive virtual-reality experience.

Display device 100 includes an on-board computing system 108 configuredto render the computerized display imagery, which is provided to rightand left display windows 106 via right-eye and left-eye optical systems102. Computing system 108 is configured to send appropriate controlsignals to right display window 106R that cause the right display windowto form a right display image. Likewise, the computing system 108 isconfigured to send appropriate control signals to left display window106L that cause the left display window to form a left display image.The wearer of the display device 100 views the right and left displayimages with right and left eyes, respectively. When the right and leftdisplay images are composed and presented in an appropriate manner, thewearer experiences the perception of virtual imagery—i.e., one or morevirtual objects at specified positions, and having specified 3D contentand other display properties. Such virtual imagery may have any desiredcomplexity; it may, for example, comprise a totally virtual scene havingboth foreground and background portions, or one of foreground andbackground to the exclusion of the other.

Operation of the display device 100 is additionally or alternativelycontrolled by one or more computing devices (e.g., remote from thedisplay device 100) in communication with the display device 100. Thecomputing system 108 may include a logic subsystem and a storagesubsystem, as discussed in more detail below with respect to FIG. 11.

The computing system 108 is in communication with various sensors andvision system components of the display device 100 to provideinformation to the computing system 108. Such sensors may include, butare not limited to, position-sensing componentry 110, a world-facingvision system 112, and a wearer-facing vision system 114. Theposition-sensing componentry 110 is usable by the computing system 108to determine the position and orientation of the display device 100 inan appropriate frame of reference. In some implementations, theposition-sensing componentry 110 provides a six degrees-of-freedom(6DOF) estimate of the three Cartesian coordinates of the display systemplus a rotation about each of the three Cartesian axes. To this end, theposition-sensing componentry 110 may include any, some, or each of anaccelerometer, gyroscope, magnetometer, and global-positioning system(GPS) receiver. The output of the position-sensing componentry 110 isused to map the position, size, and orientation of virtual displayobjects onto the right and left display windows 106.

The world-facing machine vision system 112 may include one or more of acolor or monochrome flat-imaging camera, a depth-imaging camera, and aninfrared projector. The term ‘camera’ refers herein to anymachine-vision component having at least one optical aperture and sensorarray configured to image a scene or subject. The depth-imaging cameramay be configured to acquire a time-resolved sequence of depth maps of ascene or subject. In some implementations, discrete flat-imaging anddepth-imaging cameras may be arranged with parallel optical axesoriented in the same direction. In some implementations, image or videooutput from the flat-imaging and depth-imaging cameras may beco-registered and combined into a unitary (e.g., RGB+depth) datastructure or stream. In examples in which depth-imaging camera is asuitably configured time-of-flight depth-imaging camera, a data streamrepresenting both depth and brightness (e.g., IR+depth) may be availableby combining outputs differing in phase.

The infrared projector is configured to emit infrared alignment light tothe physical space. The infrared alignment light may be reflected fromthe physical space back to the display device 100 and imaged by a cameraof each of the left-eye and right-eye optical systems 102R and 102L.

The world-facing vision system 112 may be configured to measureenvironmental attributes of a physical space surrounding display device100. In some examples, the computing system 108 may use suchenvironmental data to determine the position and orientation of thedisplay device 100, calibrate the left-eye optical system 102L with theright eye optical system 102R to align a virtual image presented to aleft eye of the user by the left-eye optical system 102L with a virtualimage presented to a right eye of the user by the right eye opticalsystem 102R, align the virtual images presented by the left-eye opticalsystem 102L and the right eye optical system 102R with the physicalspace, and/or perform other operations.

In some implementations, the display device 100 may include awearer-facing machine vision system 114. The wearer-facing machinevision system 114 may include a color or monochrome flat-imaging camera,a depth-imaging camera, and/or an infrared projector. The wearer-facingvision system 114 is configured to measure attributes of a wearer ofdisplay device 100. In some examples, such attribute data is used bycomputing system 108 to calibrate the left-eye optical system 102L withthe right-eye optical system 102R, as well as to determine a position ofthe wearer's eye(s), a gaze vector, a gaze target, a pupil position,head orientation, eye gaze velocity, eye gaze acceleration, change inangle of eye gaze direction, and/or any other suitable eye trackinginformation.

In some implementations, the computing system 108 may include anobject-recognition engine configured to compare objects resolved by thevision systems 112 and 114 to a plurality of objects stored in adatabase or defined heuristically, and to identify a match. Theobject-recognition engine 116 may be employed to calibrate the left-eyeoptical system 102L with the right-eye optical system 102R. Further, insome implementations, the object-recognition engine 116 may be employedto align virtual images generated by the left-eye optical system 102Lwith the right-eye optical system 102R with the physical space.

FIGS. 2A-2B show an example optical system 200 in simplified form. Theoptical system 200 is an example of a system that may be used as theleft-eye optical system 102L and the right-eye optical system 102R ofthe display device 100 of FIG. 1, and/or with any other suitablenear-eye display device. FIG. 2A shows the optical system 200 providingimage light 201 to a user eye 202, while FIG. 2B shows the opticalsystem 200 emitting or receiving alignment light 203, depending upon thealignment method used. The alignment light 203 is used to calibrate theoptical system 200 with a corresponding right-eye optical system, asdiscussed in further detail below.

As shown in FIG. 2A, an image source 204 outputs the image light 201 toa beam splitter 206. The image source 204 may take any suitable form,including but not limited to, a liquid crystal display (LCD) or liquidcrystal on silicon (LCOS) display. The image source 204 may employ anysuitable backlight or other illumination source. In one example, theimage source 204 may include one or more laser light sources (e.g.,laser diodes) to provide spatially coherent image light 201 to the beamsplitter 206. A laser has a narrow linewidth (e.g., emits light at asingle wavelength) that may produce little or no perceptible rainboweffect when diffracted by a hologram. The image source 204 may providethe image light 201 to the beam splitter 206 in any suitable manner. Insome implementations, the image source 204 may provide the image light201 to the beam splitter 206 at a fixed angle of incidence. In otherimplementations, the image source 204 may vary the angle of incidence atwhich the image light 201 is provided to the beam splitter 206.

The image light 201 travels along an optical path from the image source204, through the beam splitter 206, and to the lens 208. Any suitabletype of beam splitter 206 may be used, including but not limited to, adielectric mirror, a prism cube, and a polarizing beam splitter. In someimplementations, the beam splitter 206 may be omitted from the opticalsystem 200, and the image light 201 may travel directly from the imagesource 204 to the lens 208, or may travel through a different opticalcomponent.

The lens 208 is configured to direct the image light 201 at a suitableangle to enter a wave guide 210. In other implementations, the opticalsystem 200 may additionally or alternatively include an input couplingprism, embossed grating, volume hologram, slanted diffraction grating,or other coupling structure.

The image light 201 may propagate through the wave guide 210 by totalinternal reflection until it exits the wave guide 210 at alight-deflecting optical component 212. The wave guide 210 may take anysuitable form. In the illustrated implementation, the wave guide 210 hasa wedge shape. In other implementations, the wave guide 210 may have amore uniform thickness.

The light-deflecting optical component 212 is positioned adjacent thewave guide 210 in field of view of the user eye 202. In one example, thelight-deflecting optical component 212 includes a holographic opticalcomponent. In another example, the light-deflecting optical component212 includes an embossed grating. In yet another example, thelight-deflecting optical component 212 includes a Fresnel lens. Thelight-deflecting optical component 212 may be configured to deflectdifferent light beams in different directions in any suitable manner.

In the illustrated examples, the light-deflecting optical component 212is described in terms of a holographic optical component, but may takeother forms in other examples. The holographic optical component 212comprises a holographic film that is applied to at least a portion ofthe display windows 106 of display device 100 of FIG. 1. The holographicoptical component 212 may be located in any suitable position relativeto the other components of the optical system 200. Further, theholographic optical component 212 may have any suitable shape and/ororientation. The holographic optical component 212 includes an imagehologram 214 and an alignment hologram 216. In some implementations, theimage hologram 214 and the alignment hologram 216 are recorded in a sameor spatially overlapping position(s) on the holographic opticalcomponent 212, while these holograms may have different locations (e.g.formed in different layers) other examples.

The image hologram 214 is configured to diffract the image light 201 tothe user's eye 202 to produce a virtual image. In this manner, the imagelight 201 travels along a projection beam path extending between thebeam splitter 206 and the light-deflecting optical component 212

Furthermore, the alignment hologram 216 is configured to redirectalignment light used to align the left-eye and right-eye opticalsystems. As such, the optical system 200 further includes an alignmentoptical component 218. In various implementations, and as shown in FIG.2B, the alignment optical component 218 is configured to provide orreceive the alignment light 203. In some implementations, the alignmentoptical component 218 includes a camera configured to acquire analignment image from received alignment light 203. In suchimplementations, the alignment light 203 enters the optical system 200via the alignment hologram 216, which diffracts the alignment light intothe wave guide 210 at a suitable angle for the alignment light 203 tototally internally reflect through the wave guide 210. The alignmentlight 203 propagates through the wave guide 210, and exits the waveguide 210 toward the lens 208. The lens 208 directs the alignment light203 into the beam splitter 206, which in this example is configured todirect the alignment light 203 to the camera to acquire an alignmentimage. In this manner, the alignment light 203 travels along analignment beam path extending between the beam splitter 206 and thelight-deflecting optical component 212.

The beam splitter 206 may direct light in different directions basedupon any suitable optical characteristics that differ between the imagelight 201 and the alignment light 203. Examples include, but are notlimited to, wavelength and polarization state. Where the beam splitteris a polarizing beam splitter, one or more polarizers (not shown) may beincluded in optical system 200.

In other implementations, the alignment optical component 218 includesan infrared projector configured to emit infrared alignment light 203.The infrared alignment light 203 travels through the optical system 200along the same alignment beam path, but in the opposite direction,relative to implementations in which the alignment optical component 218includes a camera. In such an implementation, the infrared projectoremits the infrared alignment light 203 toward the beam splitter 206. Thebeam splitter 206 directs the infrared alignment light 203 through thelens 208 and into the wave guide 210. The infrared alignment light 203propagates through the waveguide to the alignment hologram, whichredirects the light into the physical environment. In implementationswhere the alignment optical component 218 emits infrared light, thedisplay device may further comprise an infrared camera (e.g., a depthcamera of either of the vision systems 112 or 114 of FIG. 1) configuredto capture images of the infrared alignment light 203 reflected from thephysical space to form an infrared alignment image, which is atwo-dimensional image or a depth image.

Thus, the optical system 200 defines a common optical path extendingbetween the beam splitter 206 and the holographic optical component 212along which both the image light 201 travels along the projection beampath and the alignment light 203 travels along the alignment beam path.

Because the image light 201 and the alignment light 203 share a commonoptical path extending between the beam splitter 206 and the holographicoptical component 212, an alignment light beam directed to or from thealignment optical component 218 passes through the holographic opticalcomponent 212 at a same position as an image light beam emitted from theimage source 204. This optical relationship is true for each pixel ofthe holographic optical component 212. As a result, a virtual image thatis provided to the user eye 202 is distorted in the same manner as theimage acquired by the camera.

FIGS. 2A-2B also schematically show a computing system 220 that controlsoperation of the optical system 200. The computing system 220 maycorrespond to the computing system 108 of FIG. 1, and may control bothleft-eye and right-eye optical systems. The computing system 220 isconfigured to, among other operations, align the right-eye image source204 with a corresponding image source of a left-eye optical system, andcontrol the image source 204 to provide a virtual image to the userright eye 202 that is aligned with a virtual image provided to the userleft eye.

The computing system 220 is configured to calibrate the image sources ofeach optical system based on alignment images generated from thealignment light 203, for example, to help compensate for variousdistortions in either of the optical systems due to defamation e.g.,twisting or bending), and thus to align the virtual images provided toeach user eye. For example, the computing system 220 may be configuredto determine positional offsets (e.g., along the X axis and/or the Yaxis of the field of view) of the virtual images provided by the imagesource of left-eye optical system and the image source of the righteye-optical system. Further, the computing system 220 is configured tocalibrate these image sources based on the determined positional offsetsin order to align the left-eye and right-eye virtual images. In anotherexample, the computing system 220 may be configured to adjust one ormore of the image sources to distort (e.g., stretch, shrink) at least aportion of the virtual image produced by the image source to compensatefor any deformations of the associated optical system. The computingsystem 220 may be configured to adjust any suitable image productionparameter during the calibration process to align the virtual images.

FIGS. 3-9 show various implementations of near-eye display deviceshaving different arrangements of alignment optical components foraligning images produced by left-eye and right-eye optical systems in astereoscopic display system. The various implementations are shownproviding and/or receiving various light rays. Note that the light raysare shown for purposes of illustration, are not drawn to scale, and maynot be an accurate representation of light paths used by animplementation.

FIG. 3 shows a display device 300 including a left-eye optical system302L and a right-eye optical system 302R. The left-eye optical system302L includes an alignment optical component in the form of avisible-light camera 304L, and the right-eye optical system 302R alsoincludes an alignment optical component in the form of a visible-lightcamera 304R. The left-eye optical system 302L further includes aholographic optical component 306L configured to direct ambientalignment light 308L from a physical space 310 to the camera 304L. Thecamera 304L is configured to acquire an alignment image of the physicalspace 310 based on the alignment light 308L. Further, the holographicoptical component 306L is configured to direct image light from theimage source 314L to the user's left eye such that the user's left eyeviews a left-eye image.

Likewise, the right-eye optical system 302R includes a holographicoptical component 306R configured to direct ambient alignment light 308Rfrom the physical space 310 to the camera 304R. The camera 304R isconfigured to acquire an alignment image of the physical space 310 basedon the alignment light 308R. Further, the holographic optical component306R is configured to direct image light from the image source 314R tothe user's right eye such that the user's right eye views a right-eyeimage.

A computing system 312 is configured to calibrate an image source 314Lof the left-eye optical system 302L and an image source 314R of theright-eye optical system 302R based on the alignment image acquired bythe camera 304L and the alignment image acquired by the camera 304R inorder to align the left-eye image and the right-eye image.

The display device 300 also includes an outward-facing camera 316configured to acquire an alignment image of the physical space 310 fromlight that does not travel through either the left-eye or the right-eyeoptical system (e.g., it is external to these two optical systems). Thisalignment image also may be referred to as an external alignment image,as it is acquired via a camera that is external to the left-eye andright-eye optical systems. In other words, the outward-facing camera 316acquires the external alignment image from light received directly fromthe physical space 310. The outward-facing camera 316 may be avisible-light camera configured to acquire the external alignment imagefrom ambient light, or may take any other suitable form. Theoutward-facing camera 316 may be positioned in any suitable location onthe display device 300, for example, at a location between a left eyedisplay and a right eye display. In one example, the outward-facingcamera 316 is representative of a camera included in the world-facingvision system 112 of the display device 100 of FIG. 1.

As the outward-facing camera 316 is configured to acquire the externalalignment image from ambient light that does not travel through eitherof the optical systems 302L, 302R, but is instead collected directlyfrom the physical space 310, the external alignment image is free of anydistortion caused by deformations of either of the optical systems 302L,302R. The external alignment image may include an area of the physicalspace 310 that at least partially overlaps an area of the physical space310 included in each of the alignment images acquired by the cameras304L, 304R.

The computing system 312 is configured to align a left-eye imageproduced by the image source 314L and a right-eye image produced by theimage source 314R with the physical space 310 based on a calibration ofthe alignment image acquired by the camera 304L, the alignment imageacquired by the camera 304R, and the external alignment image acquiredby the outward-facing camera 316. For example, the computing system 312may be configured to align virtual objects in the left-eye image and theright-eye image with one or more real-world features of the physicalspace 310 based on a spatial registration between the alignment images.

The computing system 312 may be configured to perform any suitablemachine vision algorithm to spatially register the images. Non-limitingexamples of machine-vision algorithms that may be employed by thecomputing system 312 include, but are not limited to, Online ContinuousStereo Parameter Estimation, a Kalman Filter that incorporates aplurality of error metrics including bundle adjustment, epipolarconstraints and trilinear constraints, and monocular visual simultaneouslocalization and mapping (SLAM). In some implementations, the computingsystem 312 may employ an object-recognition engine to perform one ormore of these algorithms, such as the object-recognition engine 116 ofFIG. 1. The computing system 312 may perform any of these and/or anyother suitable machine vision operations as part of the alignmentprocess.

FIG. 4 shows another example display device 400. The display device 400differs from the display device 300 in that the display device 400includes visible-light cameras to acquire alignment images of the userinstead of the physical space. An outward-facing camera external to theleft and right eye optical systems may be included, but is omitted inFIG. 4 for clarity.

The display device 400 includes a left-eye optical system 402L and aright-eye optical system 402R. The left-eye optical system 402L includesan alignment optical component in the form of a visible-light camera404L, and the right-eye optical system 402R includes an alignmentoptical component in the form of a visible-light camera 404R. Theleft-eye optical system 402L includes a holographic optical component406L configured to direct alignment light 408L reflected from a portionof the face of the user 410 to the camera 404L for imaging. Theholographic optical component 406L also is configured to direct imagelight from the image source 414L to the user's left eye such that theuser's left eye views a left-eye image. Likewise, the right-eye opticalsystem 402R includes a holographic optical component 406R configured todirect alignment light 408L reflected from a portion of the face of theuser 410 to the camera 404L for imaging. The holographic opticalcomponent 406R also is configured to direct image light from the imagesource 414R to the user's right eye such that the user's right eye viewsa right-eye image. A computing system 412 is configured to calibrate animage source 414L of the left-eye optical system 402L and an imagesource 414R of the right-eye optical system 402R to align the left-eyeand right-eye images provided to the user's left and right eyes based onthe alignment image acquired by the camera 404L and the alignment imageacquired by the camera 404R.

FIG. 5 shows another display device 500. The display device 500 differsfrom the previously described configurations in that the display device500 includes an infrared projector that emits infrared light to aphysical space. The infrared light is reflected back to infrared camerasof the optical systems to align left-eye and right-eye images providedby the optical systems to the user's eyes.

The display device 500 includes a left-eye optical system 502L and aright-eye optical system 502R. The left-eye optical system 502L includesan alignment optical component in the form of an infrared depth camera504L, and the right-eye optical system 502R also includes an alignmentoptical component in the form of an infrared depth camera 504R. Thedisplay device 500 includes an infrared projector 518 configured to emitinfrared light 520 into physical space 510. The infrared projector 518may be configured to emit any suitable light. In one example, theinfrared projector 518 emits infrared light in a pattern that is imagedby structured-light infrared cameras. In another example, the infraredprojector 518 emits the infrared light as a pulse that is imaged bytime-of-flight infrared cameras. In another example, the infraredprojector 518 emits constant infrared light that is imaged by a flatinfrared camera.

The left-eye optical system 502L includes a holographic opticalcomponent 506L configured to direct infrared alignment light 508Lreflected from the physical space 510 to the depth camera 504L forimaging by the camera 504L. Further, the holographic optical component506L is configured to direct image light from the image source 514L tothe user's left eye such that the user's left eye views a left-eyeimage. Likewise, the right-eye optical system 502R includes aholographic optical component 506R configured to direct infraredalignment light 508R reflected back from the physical space 510 forimaging by camera 504R. Further, the holographic optical component 506Ris configured to direct image light from the image source 514R to theuser's right eye such that the user's right eye views a right-eye image.A computing system 512 is configured to calibrate an image source 514Lof the left-eye optical system 502L and an image source 514R of theright-eye optical system 502R to align the left-eye and right-eye imagesprovided to the user's left and right eyes based on the infraredalignment image acquired by the camera 504L and the infrared alignmentimage acquired by the camera 504R.

The display device 500 optionally includes an outward-facing depthcamera 516 configured to acquire an infrared external depth alignmentimage of the physical space 510 from infrared light that does not travelthrough the optical systems. The outward-facing depth camera 516 isconfigured to acquire the infrared external alignment image frominfrared light emitted by the infrared projector 518 that is reflecteddirectly back from the physical space 510. The outward-facing depthcamera 516 may be positioned in any suitable location on the displaydevice 500. In one example, the outward-facing depth camera 516 isrepresentative of a camera included in the world-facing vision system112 of the display device 100 of FIG. 1.

The outward-facing depth camera 516 acquires the external infraredalignment image from infrared light that does not travel through eitherof the optical systems 502L, 502R, but is instead collected directlyfrom the physical space 510. As such, the external alignment image isfree of any distortion caused by deformations of either of the opticalsystems 502L, 502R. The external alignment image may include an area ofthe physical space 510 that at least partially overlaps an area of thephysical space 310 included in each of the infrared alignment imagesacquired by the depth cameras 504L, 504R.

The computing system 512 is configured to align a left-eye imageproduced by the image source 514L and a right-eye image produced by theimage source 514R with the physical space 510 based on a calibration ofthe infrared alignment image acquired by the depth camera 504L, thealignment image acquired by the depth camera 504R, and the infraredexternal alignment image acquired by the outward-facing depth camera516. For example, the computing system 512 may be configured to alignvirtual objects in the left-eye image and the right-eye image withreal-world features of the physical space 510 based on a spatialregistration between the alignment images.

In some implementations, the computing system 512 may employ anobject-recognition engine, such as the object-recognition engine 116 ofFIG. 1, to register virtual objects in the left-eye and right-eye imageswith real-world objects in the physical space. The computing system 512may perform any suitable machine vision operations as part of thecalibration process.

FIG. 6 shows a display device 600. The display device 600 differs fromthe above described configurations in that the display device utilizesinfrared cameras configured to acquire infrared alignment images of theuser instead of the physical space. An outward-facing camera external tothe left and right eye optical systems may be included, but is omittedin FIG. 6 for clarity.

The display device 600 includes a left-eye optical system 602L and aright-eye optical system 602R. The left-eye optical system 602L includesan alignment optical component in the form of a depth camera 604L orother infrared camera, and the right-eye optical system 602R includes analignment optical component in the form of a depth camera 604R or otherinfrared camera. The display device 600 includes an infrared projector618 configured to emit infrared light 620 towards at least a portion ofa user 610 (e.g., a portion of the user's face). The left-eye opticalsystem 602L includes a holographic optical component 606L configured todirect infrared alignment light 608L reflected back from at least aportion of the user 610 to the depth camera 604L for imaging a portionof the user's face. The holographic optical component 606L also isconfigured to direct image light from the image source 614L to theuser's left eye such that the user's left eye views a left-eye image.Likewise, the right-eye optical system 602R includes a holographicoptical component 606R configured to direct infrared alignment light608R reflected back from at least a portion of the user 610 to the depthcamera 604R for imaging a portion of the user's face. Further, theholographic optical component 606R also is configured to direct imagelight from the image source 614R to the user's right eye such that theuser's right eye views a right-eye image.

A computing system 612 is configured to calibrate an image source 614Lof the left-eye optical system 602L and an image source 614R of theright-eye optical system 602R to align left-eye and right-eye imagesprovided to the user's left and right eyes based on the depth alignmentimage acquired by the depth camera 404L and the depth alignment imageacquired by the depth camera 404R.

FIG. 7 shows a display device 700. The display device 700 differs fromthe above described configurations in that the display device 700includes left-eye and right-eye optical systems including infraredprojectors that emit infrared light into a physical space in front ofdisplay device. Further, the display device 700 includes anoutward-facing depth camera or other infrared camera configured toacquire depth alignment images or other infrared alignment images frominfrared light emitted from both of the infrared projectors andreflected back from the physical space.

The display device 700 includes a left-eye optical system 702L and aright-eye optical system 702R. The left-eye optical system 702L includesan alignment optical component in the form of an infrared projector 704Lconfigured to emit infrared alignment light 708L according to a firststructured light pattern. Likewise, the right-eye optical system 702Rincludes an alignment optical component in the form of an infraredprojector 704R configured to emit infrared alignment light 708Raccording to a second structured light pattern that differs from thefirst structured light pattern of the infrared alignment light 708L. Theleft-eye optical system 702L includes a holographic optical component706L configured to direct the infrared alignment light 708L towardphysical space 710. Further, the holographic optical component 706L isconfigured to direct image light from the image source 714L to theuser's left eye such that the user's left eye views a left-eye image.Likewise, the right-eye optical system 702R includes a holographicoptical component 706R configured to direct the infrared alignment light708R toward the physical space 710. Further, the holographic opticalcomponent 706R also is configured to direct image light from the imagesource 714R to the user's right eye such that the user's right eye viewsa right-eye image.

The display device 700 additionally includes an outward-facing depthcamera 716 or other infrared camera configured to acquire one or moredepth alignment images or infrared alignment images of the physicalspace 710 based on the infrared alignment light 708L and the infraredalignment light 708R reflected back from the physical space 710 to theoutward-facing depth camera 716. The outward-facing depth camera 716 maybe positioned in any suitable location on the display device 700. In oneexample, the outward-facing depth camera 716 is representative of acamera included in the world-facing vision system 112 of the displaydevice 100 of FIG. 1.

A computing system 712 is configured to calibrate an image source 714Lof the left-eye optical system 702L and an image source 714R of theright-eye optical system 702R to align left-eye and right-eye imagesprovided to the user's left and right eyes based on the one or moreinfrared alignment images acquired by the outward-facing depth camera716. In one example, the computing system 712 is configured todifferentiate between the two optical systems based on the differentstructured light patterns emitted by the different infrared projectorsof the different optical systems. In another example, the two infraredemitters of the two optical systems emit infrared light according to thesame pattern, and the computing system 712 is configured to calibratethe image sources of the two optical systems in the same manner asdescribed above with reference to the two visible-light cameraconfiguration of the display device 300.

FIG. 8 shows a display device 800. The display device 800 differs fromthe above described configurations in that the display device 800includes left-eye and right-eye optical systems including infraredprojectors that emit infrared light inwardly towards the user instead ofoutwardly toward the physical space. Further, the display device 800includes an inward-facing depth camera or other infrared cameraconfigured to acquire depth alignment images or other infrared alignmentimages from infrared light emitted from both of the infrared projectorsand reflected back from the user.

The display device 800 includes a left-eye optical system 802L and aright-eye optical system 802R. The left-eye optical system 802L includesan alignment optical component in the form of an infrared projector 804Lconfigured to emit infrared alignment light 808L according to a firststructured light pattern. Likewise, the right-eye optical system 802Rincludes an alignment optical component in the form of an infraredprojector 804R configured to emit infrared alignment light 808Raccording to a second structured light pattern that differs from thefirst structured light pattern of the infrared alignment light 808L. Theleft-eye optical system 802L includes a holographic optical component806L configured to direct the infrared alignment light 808L towards atleast a portion of a user 810. Further, the holographic opticalcomponent 806L is configured to direct image light from the image source814L to the user's left eye such that the user's left eye views aleft-eye image. Likewise, the right-eye optical system 802R includes aholographic optical component 806R configured to direct the infraredalignment light 808R towards at least a portion of the user 810.Further, the holographic optical component 806R is configured to directimage light from the image source 814R to the user's right eye such thatthe user's right eye views a right-eye image.

The display device 800 also includes an inward-facing depth camera 816configured to acquire one or more depth alignment images or otherinfrared alignment images based on the infrared alignment light 808L andthe infrared alignment light 808R reflected back from the user 810 tothe inward-facing depth camera 816. The inward-facing depth camera 816may be positioned in any suitable location on the display device 800. Inone example, the inward-facing depth camera 816 is representative of acamera included in the wearer-facing vision system 114 of the displaydevice 100 of FIG. 1.

A computing system 812 is configured to calibrate an image source 814Lof the left-eye optical system 802L and an image source 814R of theright-eye optical system 802R to align left-eye and right-eye imagesprovided to the user's left and right eyes based on the one or moredepth alignment images or the other infrared alignment images acquiredby the inward-facing depth camera 816. In one example, the computingsystem 812 is configured to differentiate between the two opticalsystems based on the different structured light patterns emitted by thedifferent infrared projectors of the different optical systems. Inanother example, the two infrared emitters of the two optical systemsemit infrared light according to the same pattern, and the computingsystem 812 is configured to calibrate the image sources of the twooptical systems in the same manner as described above with reference tothe two visible-light camera configuration of the display device 300.

FIGS. 9A-9B show a display device 900. The display device 900 differsfrom the above described configurations in that the display device 900includes left-eye and right-eye optical systems including infraredprojectors that emit infrared light into a physical space in front ofthe display device 900 from a user's perspective according to a timemultiplexing scheme. Further, the display device 900 includes anoutward-facing depth camera or other infrared camera configured toacquire depth alignment images or other infrared alignment images frominfrared light emitted from either of the infrared projectors andreflected back from the physical space based on the time multiplexingscheme. The display device 900 includes a left-eye optical system 902Land a right-eye optical system 902R. The left-eye optical system 902Lincludes an alignment optical component in the form of an infraredprojector 904L configured to emit infrared alignment light 908Laccording to a time multiplexing scheme. For example, as shown in FIG.9A, at time T1, the infrared projector 904L emits infrared alignmentlight 908L to a physical space 910 via a holographic optical component906L. Likewise, the right-eye optical system 902R includes an alignmentoptical component in the form of an infrared projector 904R configuredto emit infrared alignment light 908R according to the time multiplexingscheme. For example, as shown in FIG. 9B, at time T2 subsequent to timeT1, the infrared projector 904R emits infrared alignment light 908R tothe physical space 910 via a holographic optical component 906R. Theholographic optical component 906L also is configured to direct imagelight from the image source 914L to the user's left eye such that theuser's left eye views a left-eye image. Likewise, the holographicoptical component 906R also is configured to direct image light from theimage source 914R to the user's right eye such that the user's right eyeviews a right-eye image.

The display device 900 further includes an outward-facing depth camera916 configured to acquire a plurality of infrared alignment mages of thephysical space 910. In particular, different infrared images may capturethe infrared alignment light 908L separately from the infrared alignmentlight 908R according to the time multiplexing scheme. The outward-facingdepth camera 916 may be positioned in any suitable location on thedisplay device 900. In one example, the outward-facing depth camera 916is representative of a camera included in the world-facing vision system112 of the display device 100 of FIG. 1.

A computing system 912 is configured to calibrate an image source 914Lof the left-eye optical system 902L and an image source 914R of theright-eye optical system 902R to align left-eye and right-eye imagesprovided to the user's left and right eyes based on the plurality ofdepth alignment images acquired by the outward-facing depth camera 916.

As the alignment images for the different optical systems are acquiredat different times, a pose of the display device may change in betweendifferent alignment images. Accordingly, the computing system 912 may beconfigured to compensate for changes in pose when calibrating the imagesources based on the alignment images, or to perform alignment when thesystem is determined to be suitably stationary (e.g. from motion sensordata).

FIG. 10 shows an example method 1000 for calibrating image sources of abinocular, near-eye display device, such as the display device 100 ofFIG. 1. At 1002, method 1000 includes acquiring, via one or morecameras, a first alignment image via alignment light directed along afirst alignment beam path through a first holographic optical componentpositioned in a field of view of a first display. For example, the firstalignment beam path may extend from the first holographic opticalcomponent through an associated optical system in which the firstholographic optical component is included.

At 1004, method 1000 includes acquiring, via one or more cameras, asecond alignment image from alignment light directed along a secondalignment beam path through a second holographic optical componentpositioned in a field of view of a second display. For example, thesecond alignment beam path may extend from the second holographicoptical component through an associated optical system in which thesecond holographic optical component is included. The direction in whichthe above described alignment light travels through the holographicoptical component and the associated optical system depends on whetheran alignment optical component of the associated optical system takesthe form of a camera or a projector.

At 1006, the method 1000 includes calibrating a first image source and asecond image source to align left-eye and right-eye virtual images basedon the first alignment image and the second alignment image. Forexample, calibrating may include determining positional offsets (e.g.,along the X axis and/or the Y axis of the field of view) of either orboth the virtual images provided by the image source of left-eye opticalsystem and the image source of the right eye-optical system, andcalibrating these image sources based on the determined positionaloffsets in order to align the left-eye and right-eye virtual images. Inanother example, calibrating may include adjusting one or more of theimage sources to distort (e.g., stretch, shrink) at least a portion ofthe virtual image produced by the image source to compensate for anydeformations of the associated optical system. Any suitable imageproduction parameter of an image source may be adjusted during thecalibration process to align the virtual images.

In some implementations, calibrating may further include aligning theleft-eye and right-eye images with one or more features of an externalimage of a physical space, wherein the term “external image” indicatesthat the image was acquired via a camera external to the left-eye andright-eye optical systems. Accordingly, at 1008, the method 1000optionally may include acquiring, via an outward-facing camera, anexternal alignment image of a physical space, and at 1010, aligning thefirst image and the second image with the one or more features in theexternal alignment image of the physical space based on a calibration ofthe first alignment image, the second alignment image, and the externalalignment image. The first image and the second image may be alignedwith the external alignment image of the physical space using anysuitable algorithm, including but not limited to those described above.

At 1012, the method 1000 includes producing, via the first image source,a first image directed along a first projection beam path through thefirst holographic optical component and out of the first display to afirst eye of the user, wherein the first projection beam path shares acommon optical path with the first alignment beam path. At 1014, themethod 1000 includes producing, via the second image source, a secondimage directed along a second projection beam path through the secondholographic optical component and out of the second display to thesecond eye of the user, wherein the second projection beam path shares acommon optical path with the second alignment beam path.

FIG. 11 schematically shows a non-limiting implementation of a computingsystem 1100 that can enact one or more of the methods and processesdescribed above. Computing system 1100 is shown simplified form.Computing system 1100 may take the form of one or more personalcomputers, server computers, tablet computers, home-entertainmentcomputers, network computing devices, gaming devices, mobile computingdevices, mobile communication devices (e.g., smart phone),virtual-reality devices, and/or other computing devices. For example,the computing system 1100 may be anon-limiting example of the computingsystem 108 of the display device 100 of FIG. 1.

Computing system 1100 includes a logic machine 1102 and a storagemachine 1104. Computing system 1100 may optionally include a displaysubsystem 1106, input subsystem 1108, communication subsystem 1110,and/or other components not shown in FIG. 11.

Logic machine 1102 includes one or more physical devices configured toexecute instructions. For example, the logic machine 1102 may beconfigured to execute instructions that are part of one or moreapplications, services, programs, routines, libraries, objects,components, data structures, or other logical constructs. Suchinstructions may be implemented to perform a task, implement a datatype, transform the state of one or more components, achieve a technicaleffect, or otherwise arrive at a desired result.

The logic machine 1102 may include one or more processors configured toexecute software instructions. Additionally or alternatively, the logicmachine 1102 may include one or more hardware or firmware logic machinesconfigured to execute hardware or firmware instructions. Processors ofthe logic machine 1102 may be single-core or multi-core, and theinstructions executed thereon may be configured for sequential,parallel, and/or distributed processing. Individual components of thelogic machine 1102 optionally may be distributed among two or moreseparate devices, which may be remotely located and/or configured forcoordinated processing. Aspects of the logic machine 1102 may bevirtualized and executed by remotely accessible, networked computingdevices configured in a cloud-computing configuration.

Storage machine 1104 includes one or more physical devices configured tohold instructions executable by the logic machine 1102 to implement themethods and processes described herein. When such methods and processesare implemented, the state of storage machine 1104 may be transformed tohold different data.

Storage machine 1104 may include removable and/or built-in devices.Storage machine 1104 may include optical memory (e.g., CD, DVD, HD-DVD,Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM,etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive,tape drive, MRAM, etc.), among others. Storage machine 1104 may includevolatile, nonvolatile, dynamic, static, read/write, read-only,random-access, sequential-access, location-addressable,file-addressable, and/or content-addressable devices.

It will be appreciated that storage machine 1104 includes one or morephysical devices. However, aspects of the instructions described hereinalternatively may be propagated by a communication medium (e.g., anelectromagnetic signal, an optical signal, etc.) that is not held by aphysical device for a finite duration.

Aspects of logic machine 1102 and storage machine 1104 may be integratedtogether into one or more hardware-logic components. Such hardware-logiccomponents may include field-programmable gate arrays (FPGAs), program-and application-specific integrated circuits (PASIC/ASICs), program- andapplication-specific standard products (PSSP/ASSPs), system-on-a-chip(SOC), and complex programmable logic devices (CPLDs), for example.

When included, display subsystem 1106 may be used to present a visualrepresentation of data held by storage machine 1104. This visualrepresentation may take the form of a graphical user interface (GUI). Asthe herein described methods and processes change the data held by thestorage machine, and thus transform the state of the storage machine,the state of display subsystem 1106 may likewise be transformed tovisually represent changes in the underlying data. Display subsystem1106 may include one or more display devices utilizing virtually anytype of technology. Such display devices may be combined with logicmachine 1102 and/or storage machine 1104 in a shared enclosure, or suchdisplay devices may be peripheral display devices. As a non-limitingexample, display subsystem 1106 may include the near-eye displaysdescribed above.

When included, input subsystem 1108 may comprise or interface with oneor more user-input devices such as a keyboard, mouse, touch screen, orgame controller. In some implementations, the input subsystem maycomprise or interface with selected natural user input (NUI)componentry. Such componentry may be integrated or peripheral, and thetransduction and/or processing of input actions may be handled on- oroff-board. Example NUI componentry may include a microphone for speechand/or voice recognition; an infrared, color, stereoscopic, and/or depthcamera for machine vision and/or gesture recognition; a head tracker,eye tracker, accelerometer, and/or gyroscope for motion detection and/orintent recognition; as well as electric-field sensing componentry forassessing brain activity.

When included, communication subsystem 1110 may be configured tocommunicatively couple computing system 1100 with one or more othercomputing devices. Communication subsystem 1110 may include wired and/orwireless communication devices compatible with one or more differentcommunication protocols. As non-limiting examples, the communicationsubsystem may be configured for communication via a wireless telephonenetwork, or a wired or wireless local- or wide-area network. In someimplementations, the communication subsystem 1110 may allow computingsystem 1100 to send and/or receive messages to and/or from other devicesvia a network such as the Internet.

FIGS. 12A-12B show another example optical system 1200. Instead ofutilizing a beam splitter, optical system 1200 includes an image sourceand an alignment optical component in a side-by-side configuration. Theoptical system 1200 is an example of a system that may be used as theleft-eye optical system 102L and the right-eye optical system 102R ofthe display device 100 of FIG. 1, and/or with any other suitablenear-eye display device. FIG. 12A shows the optical system 1200providing image light 1201 to a user eye 1202, while FIG. 12B shows theoptical system 1200 emitting or receiving alignment light 1203,depending upon the alignment method used. The alignment light 1203 isused to calibrate the optical system 1200 with a corresponding right-eyeoptical system, as discussed above.

As shown in FIG. 12A, an image source 1204 outputs the image light 1201to a lens 1208. The lens 1208 is configured to direct the image light1201 at a suitable angle to enter a wave guide 1210. The image light1201 may propagate through the wave guide 1210 by total internalreflection until it exits the wave guide 1210 at a holographic opticalcomponent 1212. The holographic optical component 1212 comprises aholographic film that is applied to at least a portion of the displaywindows 106 of display device 100 of FIG. 1. The holographic opticalcomponent 1212 includes an image hologram 1214 and an alignment hologram1216. The image hologram 214 is configured to diffract the image light1201 to the user's eye 1202 to produce a virtual image. In this manner,the image light 1201 travels along a projection beam path extendingbetween the image source 1204 and the holographic optical component 1212

Furthermore, as shown in FIG. 12B, an alignment optical component 1218is configured to provide or receive the alignment light 1203. In someimplementations, the alignment optical component 1218 includes a cameraconfigured to acquire an alignment image from received alignment light1203. In such implementations, the alignment light 1203 enters theoptical system 1200 via the alignment hologram 1216, which diffracts thealignment light into the wave guide 1210 at a suitable angle for thealignment light 1203 to totally internally reflect through the waveguide 1210. The alignment light 1203 propagates through the wave guide1210, and exits the wave guide 210 toward the lens 1208. The lens 1208directs the alignment light 1203 to the camera to acquire an alignmentimage. In this manner, the alignment light 1203 travels along analignment beam path extending between the holographic optical component1212 and the alignment optical component 1218. In this implementation,the alignment light 1203 shares a common optical path with the imagelight 1201 between the holographic optical component 1212 and the lens1208, and the alignment light 1203 and the image light 1201 travel inopposite directions.

In other implementations, the alignment optical component 1218 includesan infrared projector configured to emit infrared alignment light 1203.The infrared alignment light 1203 travels through the optical system1200 along the same alignment beam path, but in the opposite direction,relative to implementations in which the alignment optical component1218 includes a camera. In such an implementation, the infraredprojector emits the infrared alignment light 1203 through the lens 1208and into the wave guide 1210. The infrared alignment light 1203propagates through the waveguide to the alignment hologram, whichredirects the light into the physical environment. In implementationswhere the alignment optical component 1218 emits infrared light, thedisplay device may further comprise an infrared camera (e.g., a depthcamera of either of the vision systems 112 or 114 of FIG. 1) configuredto capture images of the infrared alignment light 1203 reflected fromthe physical space to form an infrared alignment image, which is atwo-dimensional image or a depth image.

A computing system 1220 is configured to calibrate the image source 1204with a corresponding image source of a left-eye optical system, andcontrol the image source 1204 to provide a virtual image to the user'sright eye 1202 that is aligned with a virtual image provided to the userleft eye. In particular, the computing system 1220 calibrates the imagesource of each optical system based on alignment images generated fromthe alignment light 1203, for example, to help compensate for variousdistortions in either of the optical systems due to deformation (e.g.,twisting or bending), and thus to align the virtual images provided toeach user eye.

In the configuration described above, the image source 1204 and thealignment optical component 1218 are oriented in a side-by-sideconfiguration in which a projection beam path of the image light 1201and the alignment beam path of the alignment light 1203 are parallel ornear parallel. Thus, as mentioned above, a beam splitter may be omittedfrom the optical system 1200, as the image light 1201 travels directlybetween the image source 1204 and lens 1208 while the alignment light1203 travels directly between the alignment optical component 1218 andthe lens 1208.

In another example, a near-eye display device comprises a left-eyeoptical system and a right-eye optical system, each of the left-eyeoptical system and the right-eye optical system comprises alight-deflecting optical component positioned in a field of view of auser eye, an image source configured to emit imaging light, and analignment optical component, a projection beam path between the imagesource and the light-deflecting optical component and an alignment beampath between the alignment optical component and the light-deflectingoptical component share a common optical path. In this example, each ofthe left-eye optical system and the right-eye optical systemalternatively or additionally may further comprises a beam splitterconfigured to split the common optical path into 1) the alignment beampath extending between the beam splitter and the alignment opticalcomponent and 2) the projection beam path extending between the beamsplitter and the image source. In this example, the alignment opticalcomponent alternatively or additionally may include a camera configuredto acquire an alignment image, and the near-eye display device mayfurther comprise a computing system configured to calibrate the imagesource of the left-eye optical system and the image source of theright-eye optical system to align a left-eye image produced from theimaging light emitted by the image source of left-eye optical systemwith a right-eye image produced from imaging light emitted by the imagesource of the fight-eye optical system based on the alignment imageacquired by the camera of the left-eye optical system and the alignmentimage acquired by the camera of the right-eye optical system. In thisexample, the camera alternatively or additionally may be a visible-lightcamera, and the alignment image alternatively or additionally may be avisible-light image. In this example, the near-eye display devicealternatively or additionally may further comprise an infrared projectorconfigured to emit infrared light, the camera may be a depth camera, theinfrared light may be reflected to the depth camera via the alignmentbeam path, and the alignment image may be an infrared light image thatincludes reflected infrared light from the infrared projector. In thisexample, the near-eye display device alternatively or additionally mayfurther comprise an outward-facing camera configured to acquire anexternal alignment image of a physical space from light that does nottravel through the left-eye optical system or the right-eye opticalsystem, and the computing system alternatively or additionally may beconfigured to align the left-eye image and the right-eye image with oneor more features in the external alignment image of the physical spacebased on a calibration of the alignment image acquired by the camera ofthe left-eye optical system, the alignment image acquired by the cameraof the right-eye optical system, and the external alignment image. Inthis example, the holographic optical component alternatively oradditionally may be configured to direct light from a portion of aphysical space viewable through the field of view from a perspective ofthe user eye to the camera. In this example, the alignment opticalcomponent alternatively or additionally may include an infraredprojector configured to emit infrared light via the alignment beam path,and the near-eye display device may further comprise a depth cameraconfigured to acquire infrared light emitted by the infrared projectorof the left-eye optical system and infrared light emitted by theinfrared projector of the right-eye optical system. In this example, thealignment beam path alternatively or additionally may be configured todirect the infrared light towards a portion of a physical space viewablethrough the field of view from a perspective of the user eye, and thedepth camera may be positioned to acquire infrared light reflected fromthe physical space. In this example, the alignment beam pathalternatively or additionally may be configured to direct the infraredlight towards a user of the near-eye display device, and the depthcamera may be positioned to acquire infrared light reflected from theuser. In this example, the infrared projector of the left-eye opticalsystem and the infrared projector of the right-eye optical systemalternatively or additionally may be configured to emit infrared lightaccording to a time multiplexing scheme. In this example, the imagesource and alignment component alternatively or additionally may bepositioned side-by-side to form the common optical path shared by thealignment beam path and the projection beam path.

In another example, a near-eye display device comprises a left-eyeoptical system and a right-eye optical system, each of the left-eyeoptical system and the right-eye optical system comprises a holographicoptical component positioned in a field of view of a user eye, an imagesource configured to emit imaging light, a camera configured to acquirean alignment image, and a beam splitter configured to split a commonoptical path extending between the beam splitter and the holographicoptical component into an alignment beam path extending between the beamsplitter and the camera, and a projection beam path extending betweenthe beam splitter and the image source, and a computing systemconfigured to calibrate the image source of the left-eye optical systemand the image source of the right-eye optical system to align a left-eyeimage produced from the imaging light emitted by the image source of theleft-eye optical system and a right-eye image produced from imaginglight emitted by the image source of the right-eye optical system basedon the alignment image acquired by the camera of the left-eye opticalsystem and the alignment image acquired by the camera of the right-eyeoptical system. In this example, the camera alternatively oradditionally may be a visible-light camera, and wherein the alignmentimage may be a visible-light image. In this example, the cameraalternatively or additionally may be a depth camera, the near-eyedisplay device may further comprise an infrared projector configured toemit infrared light, the infrared light may be reflected to the depthcamera via the alignment beam path, and the alignment image may be aninfrared light image that includes reflected infrared light from theinfrared projector. In this example, the near-eye display devicealternatively or additionally may further comprise an outward-facingcamera configured to acquire an external alignment image of a physicalspace from light that does not travel through the left-eye opticalsystem or the right-eye optical system, and the computing system may beconfigured to align the left-eye image and the right-eye image with oneor more features in the external alignment image of the physical spacebased on a calibration of the alignment image acquired by the camera ofthe left-eye optical system, the alignment image acquired by the cameraof the right-eye optical system, and the external alignment image. Inthis example, the holographic optical component alternatively oradditionally may be configured to direct light from a portion of aphysical space viewable through the field of view from a perspective ofthe user eye to the camera. In this example, the holographic opticalcomponent alternatively or additionally may be configured to directlight reflected from the user of the near-eye display device to thecamera.

In another example, a binocular calibration method for a near-eyedisplay device comprises acquiring, via a left-side camera, a left-sidealignment image from alignment light directed along a left-sidealignment beam path from a left-eye holographic optical componentpositioned in a left display, acquiring, via a right-side camera, aright-side alignment image from alignment light directed along aright-side alignment beam path from a right-eye holographic opticalcomponent positioned in a right display, calibrating a left-eye imagesource and a right-eye image source based on comparing the left-sidealignment image and the right-side alignment image, producing, via theleft-eye image source, a left-eye image directed along a left-sideprojection beam path and out of the left display via the left-eyeholographic optical component, and producing, via the right-eye imagesource, a right-eye image directed along a right-side projection beampath and out of the right display via the right-eye holographic opticalcomponent. In this example, the method alternatively or additionally mayfurther comprise acquiring, via an outward-facing camera, an externalalignment image of a physical space from light that does not travel downthe left-side alignment beam path and does not travel down theright-side alignment beam path, and aligning the left-eye image and theright-eye image with one or more features in the external alignmentimage of the physical space based on a calibration of the firstalignment image, the second alignment image, and the external alignmentimage.

It will be understood that the configurations and/or approachesdescribed herein are presented for example, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated and/ordescribed may be performed in the sequence illustrated and/or described,in other sequences, in parallel, or omitted. Likewise, the order of theabove-described processes may be changed.

The subject matter of the present disclosure includes all novel andnonobvious combinations and subcombinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

1. A near-eye display device, comprising: a left-eye optical system anda right-eye optical system, each of the left-eye optical system and theright-eye optical system comprising: a light-deflecting opticalcomponent positioned in a field of view of a user eye; an image sourceconfigured to emit imaging light; and an alignment optical component;wherein a projection beam path between the image source and thelight-deflecting optical component and an alignment beam path betweenthe alignment optical component and the light-deflecting opticalcomponent share a common optical path.
 2. The near-eye display device ofclaim 1, wherein each of the left-eye optical system and the right-eyeoptical system further comprises: a beam splitter configured to splitthe common optical path into 1) the alignment beam path extendingbetween the beam splitter and the alignment optical component and 2) theprojection beam path extending between the beam splitter and the imagesource.
 3. The near-eye display device of claim 1, wherein the alignmentoptical component includes a camera configured to acquire an alignmentimage, and wherein the near-eye display device further comprises acomputing system configured to calibrate the image source of theleft-eye optical system and the image source of the right-eye opticalsystem to align a left-eye image produced from the imaging light emittedby the image source of left-eye optical system with a right-eye imageproduced from imaging light emitted by the image source of the right-eyeoptical system based on the alignment image acquired by the camera ofthe left-eye optical system and the alignment image acquired by thecamera of the right-eye optical system.
 4. The near-eye display deviceof claim 3, wherein the camera is a visible-light camera, and whereinthe alignment image is a visible-light image.
 5. The near-eye displaydevice of claim 3, further comprising an infrared projector configuredto emit infrared light, wherein the camera is a depth camera, whereinthe infrared light is reflected to the depth camera via the alignmentbeam path, and wherein the alignment image is an infrared light imagethat includes reflected infrared light from the infrared projector. 6.The near-eye display device of claim 3, further comprising anoutward-facing camera configured to acquire an external alignment imageof a physical space from light that does not travel through the left-eyeoptical system or the right-eye optical system, and wherein thecomputing system is configured to align the left-eye image and theright-eye image with one or more features in the external alignmentimage of the physical space based on a calibration of the alignmentimage acquired by the camera of the left-eye optical system, thealignment image acquired by the camera of the right-eye optical system,and the external alignment image.
 7. The near-eye display device ofclaim 3, wherein the holographic optical component is configured todirect light from a portion of a physical space viewable through thefield of view from a perspective of the user eye to the camera.
 8. Thenear-eye display device of claim 1, wherein the alignment opticalcomponent includes an infrared projector configured to emit infraredlight via the alignment beam path, and wherein the near-eye displaydevice further comprises a depth camera configured to acquire infraredlight emitted by the infrared projector of the left-eye optical systemand infrared light emitted by the infrared projector of the right-eyeoptical system.
 9. The near-eye display device of claim 8, wherein thealignment beam path is configured to direct the infrared light towards aportion of a physical space viewable through the field of view from aperspective of the user eye, and wherein the depth camera is positionedto acquire infrared light reflected from the physical space.
 10. Thenear-eye display device of claim 8, wherein the alignment beam path isconfigured to direct the infrared light towards a user of the near-eyedisplay device, and wherein the depth camera is positioned to acquireinfrared light reflected from the user.
 11. The near-eye display deviceof claim 8, wherein the infrared projector of the left-eye opticalsystem and the infrared projector of the right-eye optical system areconfigured to emit infrared light according to a time multiplexingscheme.
 12. The near-eye display device of claim 1, wherein the imagesource and alignment component are positioned side-by-side to form thecommon optical path shared by the alignment beam path and the projectionbeam path.
 13. A near-eye display device, comprising: a left-eye opticalsystem and a right-eye optical system, each of the left-eye opticalsystem and the right-eye optical system comprising: a holographicoptical component positioned in a field of view of a user eye; an imagesource configured to emit imaging light; a camera configured to acquirean alignment image; and a beam splitter configured to split a commonoptical path extending between the beam splitter and the holographicoptical component into an alignment beam path extending between the beamsplitter and the camera, and a projection beam path extending betweenthe beam splitter and the image source; and a computing systemconfigured to calibrate the image source of the left-eye optical systemand the image source of the right-eye optical system to align a left-eyeimage produced from the imaging light emitted by the image source of theleft-eye optical system and a right-eye image produced from imaginglight emitted by the image source of the right-eye optical system basedon the alignment image acquired by the camera of the left-eye opticalsystem and the alignment image acquired by the camera of the right-eyeoptical system.
 14. The near-eye display device of claim 13, wherein thecamera is a visible-light camera, and wherein the alignment image is avisible-light image.
 15. The near-eye display device of claim 13,wherein the camera is a depth camera, wherein the near-eye displaydevice further comprises an infrared projector configured to emitinfrared light, wherein the infrared light is reflected to the depthcamera via the alignment beam path, and wherein the alignment image isan infrared light image that includes reflected infrared light from theinfrared projector.
 16. The near-eye display device of claim 13, furthercomprising an outward-facing camera configured to acquire an externalalignment image of a physical space from light that does not travelthrough the left-eye optical system or the right-eye optical system, andwherein the computing system is configured to align the left-eye imageand the right-eye image with one or more features in the externalalignment image of the physical space based on a calibration of thealignment image acquired by the camera of the left-eye optical system,the alignment image acquired by the camera of the right-eye opticalsystem, and the external alignment image.
 17. The near-eye displaydevice of claim 13, wherein the holographic optical component isconfigured to direct light from a portion of a physical space viewablethrough the field of view from a perspective of the user eye to thecamera.
 18. The near-eye display device of claim 13, wherein theholographic optical component is configured to direct light reflectedfrom the user of the near-eye display device to the camera.
 19. Abinocular calibration method for a near-eye display device, the methodcomprising: acquiring, via a left-side camera, a left-side alignmentimage from alignment light directed along a left-side alignment beampath from a left-eye holographic optical component positioned in a leftdisplay; acquiring, via a right-side camera, a right-side alignmentimage from alignment light directed along a right-side alignment beampath from a right-eye holographic optical component positioned in aright display; calibrating a left-eye image source and a right-eye imagesource based on comparing the left-side alignment image and theright-side alignment image; producing, via the left-eye image source, aleft-eye image directed along a left-side projection beam path and outof the left display via the left-eye holographic optical component; andproducing, via the right-eye image source, a right-eye image directedalong a right-side projection beam path and out of the right display viathe right-eye holographic optical component.
 20. The method of claim 19,further comprising: acquiring, via an outward-facing camera, an externalalignment image of a physical space from light that does not travel downthe left-side alignment beam path and does not travel down theright-side alignment beam path; and aligning the left-eye image and theright-eye image with one or more features in the external alignmentimage of the physical space based on a calibration of the firstalignment image, the second alignment image, and the external alignmentimage.